The Deconstruction of a Study: Toward More Effective Evaluation of Research Studies in Cognitive Social Psychology

Information literacy modules produced by academic libraries to facilitate the research process typically use the criteria of relevance, timeliness, reliability, coverage and accuracy to assess the various information resources undergraduate students use to write research reports. These same criteria are applied to the wide spectrum of research sources that may range from popular magazines to research journal articles. In the field of Cognitive Social Psychology, many research questions necessitate the use of psycholinguistic stimuli (word lists, paired-associates, sentences, stories, etc.) as their treatments. This paper investigates the ability of information literacy modules based on the standards set forth by the Association of College and Research Libraries (ACRL) to assist students in evaluating empirical studies investigating social cognitive behavior. A study of social balance schemas was deconstructed and analyzed. Using the evaluation module based on ACRL standards, this study was evaluated as relevant, reliable, authoritative, and accurate. Similarly positive assessments of the study have been reached by experts in the field of social cognitive psychology. However, the evaluation of the study using questions grounded in experimental methodology and a basic understanding of psychological theory and statistical methods proved to be contradictory. A new set of analytical questions for evaluating research studies using psycholinguistic materials was generated from the errors in the experimental study. The Association of College and Research Libraries (ACRL) has defined information literacy as those abilities required " to recognize when information is needed and the ability to locate, evaluate, and use effectively the needed information. " Toward this end, the ACRL (2003) has set forth five standards, each with its own learning outcomes and performance indicators, for developing the information literacy of college students. The five standards for an information literate student are: 1. defines and articulates the need for information 2. selects the most appropriate investigative methods of information retrieval systems for accessing the needed information 3. evaluates information and its sources critically and incorporates selected information into his or her knowledge base and value system 4. uses information effectively to accomplish a specific purpose 5. understands many of the economic, legal and social issues surrounding the use of information and accesses and uses information ethically and legally The focus of the present paper is on the third of these five standards—the evaluation of journal articles, specifically those that include experimental investigations in the area of cognitive social psychology for which the stimuli used are words, sentences, …